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ABSTRACT 

Researchers are becoming increasingly aware of the 
advantages of using multiple regression as opposed to analysis of 
variance (ANOVA) or analysis of covariance (ANCOVA) . Multiple 
regression is more versatile and does not force the researcher to 
throw away variance by categorizing intervally scaled data. 
Polynomial regression analysis offers the investigator a method of 
analyzing curvilinear relationships with relative ease through 
commonly available statistical packages such as the Statistical 
Package for the Social Sciences (SPSS). This paper explores 
polynomial regression as an alternative to ANOVA for exploring 
nonlinear relationships. Challenges in interpreting the individual 
coefficients produced by polynomial regression are examined. Nine 
tables and four figures illustrate the discussion. Appendixes A and 
illustrate SPSS output. (Contains 8 references.) (Author/SLD) 
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Abstract 

Researchers are becoming increasingly aware of the 
advantages of the using multiple regression as opposed 
to ANOVA or ANCOVA. Multiple regression is more 
versatile and does not force the researcher to throw 
away variance by categorizing intervally scaled data. 
The present paper explores polynomial regression as an 
alternative to ANOVA for exploring nonlinear 
relationships. Challenges in interpreting the 

individual coefficients produced by polynomial 
regression are examined. 
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In a seminal article, Cohen (1968, p. 426) noted 
that ANOVA and ANCOVA are special cases of multiple 
regression analysis, and argued that in this 
realization "lie possibilities for more relevant and 
therefore more powerful exploitation of the research 
data." Since that time researchers have increasingly 
recognized that conventional multiple regression 
analysis of data as they were initially collected (no 
conversion of intervally scaled independent variables 
into dichotomies or trichotomies) does not discard 
information or distort reality, and that the general 
linear model 

...can be used equally well in experimental 
or non-experimental research. It can handle 
continuous and categorical variables. It can 
handle two, three, four, or more independent 
variables... Finally, as we will abundantly 
show, multiple regression analysis can do 
anything the analysis of variance does--sums 
of squares, mean squares, F ratios--and more. 
(Kerlinger & Pedhazur, 1973, p. 3) 

One challenge is using regression to investigate 
curvilinear relationships. More and more of an 
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intervention does not always monotonically yield more 
and more of an effect. 

In conducting special education research, it is 
our desire to determine not only whether a given 
treatment is effective, but also which specific levels 
of the treatment are most effective and at what point 
our treatment becomes ineffective or even detrimental. 
We would also hope to be able to use our results to 
make predictions about the effect of our intervention 
for individuals who we believe are similar to those 
included in our test sample, but for whom we do not 
have scores on the dependent variable. 

While ANOVA can be used to determine (a) whether 
the means of the treatment groups increase or decrease 
in a linear fashion with an increase in the level of 
the independent variable; (b) whether the trend is 
linear or nonlinear; and, if the trend is nonlinear, 
(c) what degree equation is required to fit the data 
(Hinkle, Wiersma, & Jurs, 1994, p. 372), the vital 
piece of information which such an analysis cannot 
provide is where the transitions in effectiveness occur 
(because the data for the predictor variable must be 
categorized to enable us to perform the ANOVA analysis 
and because ANOVA tests only the difference between 
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group means) . The only exception to this rule is when 
we used quantative and equally-spaced levels within an 
ANOVA way (e.g., 10, 20, and 30 minutes of 
intervention. Multiple regression can be employed to 
explore where these transitions occur. As noted by 
Pedhazur 

If the data depart significantly from 
linearity, one can do a multiple regression 
analysis in which the continuous variable is 
treated as a categorical variable. All that 
such an analysis can tell, however, is 
whether there is some trend in the data. 
When it xs desired to study the nature of the 
trend, it is necessary to resort to nonlinear 
models... By an appropriate transformation, 
a model that is nonlinear in the variables 
may be reduced to a linear model ... (Pedhazur, 
193, p. 404) . 

An example of such a transformation is polynomial 
regression, in which the independent variable values 
are raised to powers. Table 1 describes commonly used 
polynomial equations involving a single predictor 
variable. 
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Insert Table 1 about here. 



In polynomial regression, the transformed 
(powered) variables are entered into a hierarchical 
multiple regression analysis. A hierarchical multiple 
regression analysis is one which is carried out in 
steps. The first-degree vector (X) is entered, 
followed sequentially by the higher-degree polynomials. 
As shown in Table 1, the first-degree polynomial tests 
for a linear relationship between our predictor 
variable (X) and the dependent variable (Y) . The 
second order polynomial testa for a quadratic 
relationship, and a third order polynomial tests for a 
cubic relationship between the independent and 
dependent variables. 

At each stage of the polynomial regression 
analysis, the increase in variance accounted for by the 
new (higher order) polynomial, above that explained by 
the lower order term(s), can be tested for statistical 
significance. Given a continuous predictor variable 
with four distinct values, the increase in variance 
accounted for at each stage is calculated as follows: 
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Linear: R 2 y .* 

Quadratic: R 2 y . x , X2 - R 2 y . x (or R 2 quadratic-R 2 linear) 

Cubic: R 2 y.x,x 2 , X3 -R 2 y.*, X2 (or R 2 cubic-R 2 quadratic) 

and the F ratio for testing the statistical 
significance of these increments in variance accounted 
for is calculated as follows: 
Linear: 

F=( (R 2 y .*) / (kl) ) / ( (1-R 2 y.<) / (N-kl-1) ) 

Quadratic : 

F=( {R 2 y. x , xr R 2 y . x ) / (kl-k2) }/( {l-R 2 y. XfX2 ) / (N-kl-1) ) 

Cubic: 

F~{R 2 y.K, X2fXr R*y. x , X2 ) / (kl-k2) / ( {l-R 2 y. x , X2/X3 ) / (N-kl-1) ) 
where N = number of subjects; 

kl = degrees of freedom of the larger R 2 / and 

k2 = degrees of freedom of the smaller R 2 . 

Although they may appear a bit complex, these 
analyses can be performed easily using readily 
available statistical packages such as SPSS. Thus 
there is no need to categorize the intervally scaled 
independent variable data, and in the process discard 
variance, in order to perform an ANOVA trend analysis. 

To facilitate discussion of the process involved 
in polynomial regression, a small heuristic data set 
will be analyzed. The analysis is performed using SPSS 



Multiple Regression Analysis of Nonlinear Relationships 

8 



commands presented in Appendix A. Tables 2 through 4 
present the data to be analyzed. 



Insert Tables 2-4 about here. 
Table 2 shows the values for the predictor 
variable (X) and a set of values for a dependent 
variable (Yl). Table 3 includes the same values for X 
but a different set of dependent variable (Y2) values. 
Table 4 shows yet another set of dependent variable 
values (Y3) which might be obtained with the same 
values of the dependent variable (X) . These are three 
different hypothetical data sets which have no 
relationship to one another. The values of the 
variables have been selected in order to demonstrate 
different possible outcomes of polynomial regression 
analysis. Figures 1 through 3 show the scatter plots 
of the three data sets. 

Insert Figures 1 through 3 about here. 

As has been made abundantly clear (Anscombe, 
1973), a quick examination of the scatter plot for a 
given set of results can provide important clues about 
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the nature of the relationship between two variables. 
Examination of the scatter plot of the values of X and 
Yl, for example, reveals that the relationship between 
X and Yl is positive and linear. Figures 2 and 3 show 
that the relationship between X and Y2 is apparently 
quadratic (there is probably one bend in the regression 
line) and the relationship between X and Y3 appears 
cubic (there are probably two bends in the regression 
line) . 

Now we turn to describing the procedures used to 
determine the prediction equation for each of these 
three sets of data. A complete analysis of all 
possible levels of the polynomial equation is performed 
for each data set (even X with Yl, which is obviously 
linear) for purposes of explanation and comparison. 
Performing the polynomial regression analysis 

The first step in polynomial regression is to 
determine the highest degree polynomial which could be 
applied with the given data set. The highest degree 
polynomial which can be used for a set of data is g-1 
where g = the number of distinct values of the 
independent variable. Whenever the highest-degree 
polynomial for a given set of data is used, R 2 =|j. 2 . 
(This relationship will be discussed later — however, 
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remember that our interest, ultimately, is not in the 
highest order polynomial, but rather the lowest order 
polynomial which adequately describes our data. To find 
the lowest order polynomial which adequately describes 
the data, all possible levels are tested in the 
preliminary polynomial regression analysis. In most 
data sets we can test many different polynomials, but 
usually we do not go beyond 4 or 5, even when we 
could.) Since there are 5 distinct levels of the 
independent variable (X) in each of the three data 
sets, the highest degree polynomial possible for any of 
the data sets is fourth-degree, or quartic. 

Once the highest-degree polynomial that can be 
tested has been determined, the next step is to compute 
the vectors for each polynomial equation. The values 
for X and Y may be entered directly into the data file, 
and the powered vectors (X 2 , X 3 , and X 4 ) can be 
generated using COMPUTE commands (see Appendix A, lines 
9-11) . 

Next, these vectors are entered into the 
polynomial regression equation hierarchically, 
beginning with the first-order polynomial (X) and 
entering at the next three steps the second-, third- 
and fourth-degree polynomials (X 2 , X 3 , and X 4 ) 
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sequentially (see, for example, Appendix A, lines 13- 
15) . 

Interpreting the results of the polynomial regression 
Data Set #1 



Insert Table 5 about here. 



Table 5 is a summary the results of the 
preliminary polynomial regression analysis for the 
relationship between X and Yl . (For complete SPSS 
output, see Appendix B.) As discussed previously, 
examination of the scatter plot of X and Yl reveals 
that the relationship between these two variables is 
strongly linear. Thus one would expect the results to 
show a statistically significant R 2 for the linear 
relationship. Furthermore, assuming a linear 
relationship between the two variables, one would also 
expect that each of the remaining steps of the equation 
(entering X 2 , X 3 , and X 4 ) would not produce 
statistically significant increases in variance 
accounted for above what is explained by the first- 
degree polynomial. An examination of Table 5 reveals 
that the results of the regression analysis conform to 
these expectations. 
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In examining r2, it is noted that R 2 y. x,X2,X3,x 4 
~ .75000, which means that 75% of the variance in the 
dependent variable (Yl) is accounted for by the fourth- 
degree polynomial of the independent variable (X) . The 
F test for R 2 is: 

F= (.-75000/4)/( 75000)7 (15-4-1) 5 =7.5 
with 4 and 10 df, p < .01. The same result will be 
found by using the Sum of Squares and Mean Square 
Residual . 

F = (30/4)/(10/(15-4-l) ) =7.5 
This is the same F ratio that would be obtained 

if the data had been analyzed using a one-way ANOVA. 

As noted by Pedhazur, 

Testing the highest-degree polynomial 
possible in a set of data is tantamount to a 
test of whether the means of the arrays are 
equal, which is equivalent to testing 
whether the means of the treatments differ 
from each other when a one-way analysis of 
variance is applied. (Pedhazur, 1982, p. 
409) 

The test of statistical significance indicates 
that there is, in fact, a statistically significant 
trend in the data. What we stl.ll do not know, however, 



Multiple Regression Analysis of Nonlinear Relationships 

13 

is what degree polynomial best fits these data. To 
find the answer to this question, one must further 
examine the results of the polynomial regression 
analysis . 

The increment in variance accounted for by each 
degree of the polynomial equation is reported in Table 
5 in the fifth' column as R 2 Change and in the seventh 
column as Sum of Squares Change. Neither of these 
numbers is produced as output from SPSS, but both are 
easy to calculate. To calculate R 2 Change, simply find 
the R 2 for the step of interest and subtract the R 2 for 
the previous step. (These values are produced in the 
SPSS Output.) For example, for the second step using 
Data Set #1, the R 2 Change is found by finding the R 2 
for Step 2, which is the R 2 for the linear and 
quadratic components, and subtracting the R 2 for Step 
1, which is the R 2 for the linear component alone. In 
this example, the R 2 Change for the second step is 
.75000 - .75000 =0. In fact, the second-, third- 

and fourth-degree polynomials each result in an 
increment of zero because the relationship between X 
and Yl is perfectly linear. 

Sum of squares ch ..ge is calculated in the same 
manner. Simply locate the Sum of Squares for the step 
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of interest and subtract the Sum of Squares for the 
previous step. In this case, for Step 2 (the test of 
the quadratic relationship), 30 - 30 = 0. As was the 
case with R 2 Change, calculation of Sums of Squares 
Change shows that the second, third and fourth steps of 
the regression equation result in zero increase in 
variance accounted for beyond that explained by the 
linear component because the relationship between X and 
Yl is perfectly linear. For this reason, the test of 
statistical significance of the increment in variance 
accounted for by higher-order equations will be 
discussed using the second data set. 
Data Set #2 

The results of the polynomial regression for the 
second data set are summarized in Table 6. As was the 
case for the first data set, the highest-degree 
polynomial which can be used for this data set is 
fourth-degree, or quartic. Table 6 shows that for the 
final step, R 2 = .54545. For the final step, F = 3.00 
with 4 and 10 df , (not statistically significant) . We 
now turn to the test of R 2 Change. 



Insert Table 6 about here 
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R 2 Change and Sum of Squares Change have been 
calculated following the same procedures described for 
the first data set, and are presented in Table 6. The 
next step is to test the statistical significance of R 2 
Change or Mean Square Change for each step. Each step 
of the polynomial regression has 1 df, therefore Sum of 
Squares Change = Mean Square Change. Thus, the F test 
for each step of the polynomial regression is 

F = Mean Squares Change/Mean Squares Residual 
with 1 df for the numerator. The df for the 
denominator of each step is the df Residual for the 
last step of the equation. For this data set, df 
Residual = 10. Thus, for the first step of Data Set 
#2, 

F = 0.300/1 = .3 
with 1 and 10 df. For the second step of the 
regression, 

F = 10.5/1 = 10.5 
with 1 and 10 df, p < .01. (Remember that each step 
has 1 df. Do not use the df reported for each step by 
the SPSS Output because these df's are cumulative, and 
we are interested in incremental and not cumulative 
effects.) These and the remaining F ratios for the 
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data set are found in the tenth column of Table 6. The 
reader can see that for Data Set #2, only the second 
step (quadratic) of the regression equation resulted in 
a statistically significant R 2 Change. 
Data Set #3 

The procedures described above for the first two 
data sets have been followed in creating the contents 
of Table 7. The polynomial regression analysis of Data 
Set #3 results in a statistically significant R 2 Change 
for the linear (p < .01) and cubic (p < .01) levels 
only. 



Insert Table 7 about here 



Performing the hierarchical regression 

After the highest order polynomial which 
produces a statistically significant increment in R 2 
has been identified, the regression is run again, this 
time using only the vectors up to and including the 
highest-order vector which produced a statistically 
significant R 2 Change (including all lower order 
vectors regardless of whether or not they produced a 
statistically significant R 2 Change) . 
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For Data Set #1, a simple linear regression of 
Yl on X can be performed because only the linear vector 
produced a statistically significant change in variance 
accounted for. For Data Set #2, vectors (X) and (X2) 
will oe entered into the final regression analysis 
because R 2 change for X2 was statistically significant 
and R 2 Change for X3 and X4 were not. Finally, for 
Data Set #3, where the highest order polynomial which 
resulted in a statistically signif : -ant increment in R 2 
was found at the third step (the cubic relationship), 
vectors (X), (X2), and (X3) will be entered. 

Impact oq Sums of Squares, df, and Mean Squares 
Since for Data Set #1, only the linear term (X) 
will be included in the final regression equation, the 
Sums of Squares and df associated with the quartic 
(X2), cubic (X3), and quadratic (X4) terms of the 
original polynomial analysis will be pooled with Sums 
of Squares and df Residual in the final regression 
analysis. Following the same principle, for Data Set 
#2, the Sums of Squares and df associated with X3 and 
X4 will be pooled with Sums of Squares and df 
Residual, and for Data Set #3, the Sums of Squares and 
df associated with X4 will be pooled with Sums of 
Squares and df Residual. 
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The result of this pooling is that, in general, 
Sums of Squares Residual will increase slightly, but df 
Residual will also increase (when compared with the 
Sums of Squares and df Residual for the final stage of 
the original polynomial regression) resulting in a 
smaller Mean Square Residual (error) . This decrease in 
Mean Square Residual is important because Mean Square 
Residual is used in the test of statistical 
significance of R 2 . Since Mean Square Residual is the 
denominator of the equation, decreasing the size of 
Mean Square Residual increases the likelihood of 
finding a statistically r ignificant F ratio. 
The regression equation 

Data Set #2 will be used as an example for 
discussion of the regression equation. Table 8 shows 
the results of the final regression analysis for Data 
Set #2. 



Insert Table 8 about here. 



In hierarchical regression, only the b weight 
associated with the highest order polynomial in the 
regression equation is tested for statistical 
significance. Thus, for this data set, only the jb 
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weight for the quadratic component is relevant. In 
other words, we are interested in a test of the squared 
semipartial correlation of the dependent variable (Y2) 
with the highest-order vector (X2) when all the other 
vectors have been partialed out from it, or, stated 
more simply, a test of the increment of variance 
accounted for by the quadratic term over and above the 
linear term. It is unnecessary to test the b for the 
linear term because of the hierarchical nature of this 
analysis. The F test for b for this data set is: 
F = t 2 

= (-0.1250/0.037268) 2 = 11.2498 
with 1 and 12 df ', p < .01. 

Given the information in Table 8, it is now 
possible to determine the equation for the regression 
of Y2 on X. The equation is: 

Y'= -.800 + 1.55(X) + (-.125) (X 2 ) 

The regression equation can now be used to 
predict the dependent variable score (Y) based on 
knowledge of the individual ' s' score on (X). Table 9 
shows Y' scores based on the regression equation. The 
regression line, along with the original data points 
for Data Set #2, is shown in Figure 4. 
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Insert Table 9 and Figure 4about here. 



As is shown in Figure 4, the regression equation 
can now be used to predict Y 1 scores given an X value 
within the range of values of the predictor variable (X 
values) included in the data set on which the 
regression equation was developed. One should not, 
however, extrapolate beyond the values of the predictor 
variable included in the formulation of this regression 
equation. This regression equation may or may not 
apply equally well to a score of 20 on the predictor 
variable. To determine the regression equation to 
predict Y given an X score of 20, it is necessary to 
include higher values of the predictor variable (at 
least a value of 20) in another study and perform 
another regression analysis. Because it has already 
been shown that, within the given range of X values, 
the relationship between X and Y is curved, it stands 
to reason that at greater values of the independent 
variable, the regression line could have even more 
bends. The same caution applies to extrapolating below 
the lowest level of the predictor variable used in 
developing the regression equation. 
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Interpreting b weights 

Interpreting polynomial regression results can 
be a bit challenging and requires a clear understanding 
of what each of the individual coefficients represents. 
The difficulty arises when one treats the powered 
vectors as if they were distinct predictor variables. 
For example, in the regression equation Y 1 = a + b^X + 
b 2 X 2 + e, 

By definition bi and b 2 measure the change in 
Y associated with each unit of change of X or 
X 2 , respectively, controlling for the effects 
of the other... But this is impossible 
because X 2 must vary with X, for X 2 is merely' 
a vector whose elements are squares of 
corresponding elements for X. (Stimson, 
Carmines & Zeller, 1978, p. 520) 
Thus the usual interpretation of b weights does not 
make sense in polynomial regression. As Stimson et al. 
(1978) explain, polynomial equations "treat as two 
variables what is only one" (p. 520) . In other words, 
Y is not a multiple linear function of two separate 
variables X and X2, it is a curvilinear function of X 
only. 
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A second difficulty in interpreting b weights 
in polynomial regression is that the magnitude of bl 
and b2 cannot be compared because their variances are 
unequal. Generally, the variance of the quadratic (X2) 
term will be much greater than that of the linear (X) 
term. Furthermore, the variance of the cubic term (X3) 
will be greater than that of the quadratic term. 
Because of the inequality of the variances, a 
comparison of the coefficients will underestimate the 
impact of the more variable term. (Stimson et al., 
1978) 

A third challenge in interpreting b weights in 
polynomial regression is presented by the fact that 
powered vectors tend to be highly correlated. As 
pointed out by Smith and Sasaki (1979), as 
multicollinearity increases, the standard errors of the 
coefficients increase. 

Several authors (Bradley & Srivastava, 1979; 
Marquardt & Snee, 1975; Smith & Sasaki, 1979) advocate 
the use of centering of the independent variable to 
reduce multicollinearity. Centering is the process of 
performing a linear transformation of X (such as X' = X 
- X) such that the independent variable has a zero 
sample mean. As Cohen (1978) points out, the R 2 (and 
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the F value for R 2 ) for a polynomial remains invariant 
over linear transformation of X. Thus centering will 
reduce the correlation between the vectors to zero and 
reduce the variance of the powered vectors without 
changing the hierarchical regression analysis. 
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Conclusion 

Polynomial regression analysis offers the 
investigator a method of analyzing curvilinear 
relationships with relative ease through the employment 
of commonly available statistical packages such as 
SPSS. 

Researchers should be thoroughly familiar with 
the mechanics of any statistical procedure they employ. 
Polynomial regression is no exception to the rule. 
While interpretation of the results of polynomial 
regression presents certain challenges and requires 
that the researcher give careful consideration to the 
coefficients and what they represent, this is no more 
than would be expected of any researcher who hopes to 
accurately interpret results. 
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Figure 3 
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Appendix A 

00006 TITLE 'OUTPUT' 

00007 DATA LIST FILE=ABC /l 

00008 X 1-2 Yl 4 Y2 7 Y3 11 

00009 COMPUTE X2=X**2 

00010 COMPUTE X3=X**3 

00011 COMPUTE X4=X**4 

00012 LIST VARIABLES=ALL/CASES=500/FORMAT=NUMBERED 

00013 REGRESSION VARIABLES=Y1 X X2 X3 X4/DESCRIPTIVES=DEFAULT/ 

00014 CRITERIA=TOLERANCE (. 000001 ) /DEPENDENT=Y1 /ENTER X/ENTER X2 

00015 /ENTER X3/ENTER X4 

00016 REGRESSION VARIABLES=Y2 X X2 X3 X4/DESCRIPTIVES=DEFAULT/ 

00017 CRITERIA=TOLERANCE (. 000001 ) /DEPENDENT=Y2/ENTER X/ENTER X2 

00018 /ENTER X3/ENTER X4 

00019 REGRESSION VARIABLES=Y3 X X2 X3 X4/DESCRIPTIVES=DEFAULT/ 

00020 CRITERIA=TOLERANCE (. 000001 ) /DEPENDENT=Y3/ENTER X/ENTER X2 

00021 /ENTER X3/ENTER X4 
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